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Abstract. We study spectra of noncommutative dynamical systems, representations of fractal groups, and 
regular graphs. We explicitly compute these spectra for five examples of groups acting on rooted 
trees, and in three cases obtain totally disconnected sets. 

Spectres de systemes dynamiques non-commutatifs et de graphes 
associes a des group es fractals 

Resume. Nous etudions les spectres de systemes dynamiques non commutatifs, de representations de groupes 
fractals, et de graphes reguliers. Nous calculons ces spectres pour cinq exemples de groupes agissant 
sur des arbres enracines, et dans trois cas obtenons des ensembles completement deconnectes. 



Version frangaise abregee 

Le but de cette note est d'etudier le spectre de systemes dynamiques non commutatifs, i.e. de systemes 
engendres par plusieurs transformations qui ne commutent pas necessairement. Elle resume les resultats 
de[I]. 

Soit G un groupe engendre par un ensemble symetrique fini S. Un systeme dynamique, note (S, X, fj,), est 
une action de G sur un espace mesure X preservant la classe [fj] de la mesure [i de X. II induit naturellement 
une representation unitaire n de G dans L 2 (X 7 [i) donnee par (ir(g)f)(x) = \J q(x) f (g" 1 x) , ou g(x) est la 
derivee de Radon-Nikodym dgn(x)/d[i(x). Le spectre de (S,X,n) est le spectre de Poperateur de type Hecke 

Hrr = ^^( S )eB(L 2 (X,fl)). 



Plus generalement, le spectre d'une representation unitaire n : G — > U(H) d'un groupe avec systeme genera- 
teur fixe est le spectre de Poperateur comme ci-dessus. 

On considere, pour cinq exemples de groupes donnes par leur action sur un arbre regulier enracine T, leur 
^ ' representation ir dans L 2 (dT, fi) ou fi est la mesure de Bernoulli uniforme. Cette representation s'approxime 
\ par les representations n n sur les sommets a distance n de la racine de T. 

Nous montrons que ir se decompose en une somme de representations 7r„ 7r„_i de dimensions finies, et 
spec(7r) = U„>o s P ec ( 7r «)- 

H n a un spectre purement ponctuel et son rayon spectral est une valeur propre. On peut decrire ex- 
plicitement ces spectres comme suit. Pour A G R, soit J(A) Pensemble de Julia du polynome quadratique 



z 2 - A : J(A) = y\±yf\±JX±j77: 



Note presentee par Jean-Pierre Serre 
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Groupe 


Spectre de H„ 


Description 
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r 
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[-i,0]U[|,l] 
[0,1] 

{l,i}ui(l±J(6))} 
{l,-i,i}ui(l± v /|±2J(f)J 


deux intervalles 
intervalle positif 

union d'un ensemble de Cantor de 
mesure nulle et de points isoles 

ensemble de Cantor de mesure nulle 


r 




identique a F 



Ces calculs impliquent l'existence de graphes a croissance polynomiale, qui sont les graphes de Schreier 
de groupes de croissance intermediaire, et dont le spectre de Poperateur de Markov est un quelconque des 
ensembles ci-dessus. Un resultat analogue est vrai pour les systemes dynamiques. 

1. Introduction 

The purpose of this note is the study of spectra of noncommutative dynamical systems (that is, systems 
generated by several transformations that do not necessarily commute). We produce several examples of 
computations of such spectra with an interesting topological structure. More details appear in pQ. 

In the classical case defined by a single aperiodic measure-preserving transformation T : X — ► X, the 
spectrum of the corresponding operator ^(U + U~ 1 ), with U € U(L 2 (X)), is [—1, 1] by Rohlin's Lemma, but 
in the noncommutative case the spectrum may have gaps, and even be a Cantor set. 

We also study spectra of infinite regular graphs and produce the first example of a regular graph whose 
spectrum is a Cantor set. 

Our dynamical systems (with associated Hecke operator) arise from actions of fractal groups on the 
boundary of the regular rooted tree on which they act. The graphs (with associated Markov operator) 
whose spectra we consider are the Schreier graphs of these groups over "parabolic subgroups". In special 
cases, these graphs are "substitutional graphs" and have polynomial growth. 

If the underlying group is amenable, then the spectrum of the the dynamical system coincides with the 
spectrum of the corresponding graph. 

The computation of the above spectra is based on operator recursions that hold for fractal groups and 
involves a 1-dimensional and 2-dimensional classical dynamical system as an intermediate step. This leads 
to the appearance of Julia sets of quadratic maps in the description of the above spectra. 

Both authors wish to thank heartily Pierre de la Harpe and Alain Valette for their numerous comments 
and contributions to this note. 

2. Spectra of Dynamical Systems and Representations 

Let G be a group finitely generated by a symmetric set S. A non- commutative dynamical system, denoted 
(S,X,/i), is an action of a group G (generated by S) on a space X and preserving the measure class [/i] of 
a measure /ionI. 

Such a dynamical system gives rise to a natural unitary representation n of G in L 2 (X, ^i) given by 

(7r(g)f)(x) = yMxjfig-'x), 

where q(x) = dg/j,(x)/dfi(x) is the Radon-Nikodym derivative. The spectrum of the dynamical system 
(S, X, n) is the spectrum of the Hecke type operator 

(this terminology comes from an analogy with Hecke operators in number theory More generally, the 

spectrum of a unitary representation it : G — ► U (H) of a group with a given finite set of generators is the 
spectrum of the operator H„ as above — see 

Definition. A graph is a pair Q — (V,E) of sets (the vertices and edges ), a map a : E — > V (the start of an 
edge) and an involution 7 : E — > E (the inversion/ One defines then the end u>(e) of an edge by u>(e) = a(e). 

The degree of a vertex is deg(u) = |{e G E\ a(e) = v}\. The graph is locally finite if dcg(v) < oo for all 
v e V, and is regular if dcg(w) is constant over V . The graph is a tree if in every circuit (ei, e%, . . . , e n ) 
of edges with cj(e;) = a(e;+i) (indices modulo n) there is a reduction, i.e. an i with e7 = e^+i. A graph 
morphism is a pair of maps between the vertex and edge sets that commute with a and '. 



Let E = {1, . . . , d} be a finite set of cardinality d. The d- regular rooted tree Td is the graph with vertex set 
E*, edge set E* x S x {±}, and maps a(cr, s, +) = a, a(cr, s, — ) = as and (a, s, +) = (a, s, — ). Its boundary 
is dT = E N , the set of infinite sequences over E. 

Suppose now that G acts by automorphisms on a rooted tree T. This action extends to a continuous 
action on the boundary of the tree, which is a compact, totally disconnected space. The uniform measure on 
E N is the measure fj, defined on the cylinders crE N (for a £ E*) by /i(crE*) = cH CT L It is G-invariant, and is 
the unique invariant measure if G acts transitively on each level E™ of the tree, or equivalently if (S, E N , 
is ergodic. Note that all other nondegenerate Bernoulli measures are quasi- invariant. 

For each ngN, let ir n be the unitary representation (of finite dimension d n ) of G in £ 2 (E n ) induced by 
the action of G on the n-th level, and let it be the unitary representation of G in L 2 (E N ,/i). Then 7r„ is a 
subrepresentation of ir n +i and of 7r, and the spectra of 7r n converge to that of 7r: 

spec(7r) = (J spec(7r„). 

3. Spectra of Graphs 
Let Q be a locally finite graph. The Markov operator of Q is the operator M on £ 2 (V) given by 

e£E: a(e)—v 

The operator M is the transition operator for the simple random walk on Q. 

The spectral properties of M are of great importance; for instance, a theorem of Kesten ^Tj (extended 
by Dodziuk and others) claims that a graph Q of bounded degree is amenable if and only if 1 £ spec(M). 
(See more on amenability in Section 0) 

Definition. Let G be a group finitely generated by a symmetric set S, and let H be any subgroup. The 
Schreier graph of G with respect to H is the graph S(G,H,S) with vertex set G/H and edge set S x G/H, 
and maps a{s,gH) = gH and (s,gH) = (s _1 , sgH) . It has a natural base-point H . 

In Subsection \4-3\ the graphs will be labelled. This is simply done by assigning to each edge {s,gH) the 
labeling s. 

If H = 1, we obtain the usual Cayley graph of (G, S). Note that S(G, H, S) is an IS'l-regular graph, but 
its automorphism group does not necessarily act transitively on its vertices. Indeed, basically any regular 
graph is a Schreier graph ^1 Theorem 5.4]. 

G acts by left-multiplication on G/H, the vertex set of S(G, H, S). The corresponding unitary represen- 
tation in £ 2 (G/H) is the quasi-regular representation Pg/h- 

Suppose now that G acts on a rooted tree 7^. Fix the ray e = dd ■ ■ ■ £ E N . Let P n be the stabilizer of 
vertex at level n of this ray, and let P = HneN be the stabilizer of the infinite ray e (it is called a parabolic 
subgroup). We write M n the Markov operator of S(G, P n , S) and M the Markov operator of S(G, P, S); 
they are the Hecke type operators associated with the action of G on E" and E N . 

4. Fractal Groups and Substitutional Graphs 

Let G be a group acting on a tree T, and let H = Hsgs Stabc(s) be the first level stabilizer. Restricting 
the action of H to each subtree spanned by sE* gives an embedding 

ip:H^> Aut(T) s . 

Definition. The group G is fractal if tp is a subdirect embedding of H in G s , i.e. if ip(H) lies in G s and 
its projection on each factor is onto. 

Let now G be a group finitely generated by a symmetric set S, and let (X, xo) be a pointed space on 
which G acts. The growth of X is the function 7 : N — > N given by 

j(n) = \{x £ X : x = s\ . . . s n xo for some Sj £ S}\. 

X has polynomial growth if 7(71) < n D for some large enough D, has exponential growth if j(n) > X n for 
some small enough A > 1, and has intermediate growth in the remaining cases. The growth of G is its growth 
under its action on itself by left multiplication. 

We describe now briefly five archetypical examples of groups. 



4.1. The Groups and 0. The first group was introduced by the second author in 1980 0; both groups 
act on 72. Let a be the automorphism permuting the top two branches of and define recursively b, c, d 
by (j)(b) = (a, c), 0(c) = (a, d) and 0(d) — (1, b). Let © be the group generated by {a, b, c, d}. 

Define also 6, c, d by 0(6) = (a,c), 0(c) = (l,d) and 0(d) = (1,6), and let © be the group generated by 
{a,b,c,d}. Clearly © contains © as the subgroup (a,bc,cd,db). 



4.2. GGS Groups. Let d be a prime number. Denote by a the automorphism of 7^ permuting cyclically 
the top d branches. Fix a sequence e = (ei, . . . , e<j-i) € (Z/d)"* -1 . Define recursively the automorphism f e 
of 7d, written t e = (a ei , . . . , a 6 " 1 - 1 , t e ), by 



te(ciCT2 • ■ • cr n ) = 



cria c "i ((72 . . • (7„) if 1 < ei < d — 1, 
ait e (a 2 ■ ■ ■ (J n ) if cr 1 = d. 



Then G e is the subgroup of Aut(7d) generated by {a,t e }. It is called a GGS group PJ. 

The following results belong to folklore (see 0): G e is an infinite group if an only if e ^ (0, . . . , 0). It is 
a torsion group if and only if e i = 0. The only three infinite GGS groups for d = 3 are as follows: 

Let r be the automorphism of % defined recursively by 0(r) = (a, l,r), and let F be the subgroup of 
Aut(7a) generated by {a,r}. It was first considered by Narain Gupta and Jacek Fabrykowski 

Define s by <j>(s) — (a, a, s), and let T be the subgroup of Aut(7a) generated by {a, s}. 

Define t by <j>(t) = (a, a -1 , t), and let T be the subgroup of Aut(7a) generated by {a, i}; it was first studied 
in the 80 's by Narain Gupta and Said Sidki [HI EJ - 

Theorem 1. The groups ©,(5 and all infinite GGS groups with d > 3 have intermediate growth. The 
Schreier graph S(G, P, S) corresponding to G = ©, © ot Q7iy GGS group has polynomial growth. 

In particular, 5(0, P, 5) and 5(0,P,5) have linear growth, while S(T,P,S), S{T,P,S) and S(T,P,S) 
have polynomial growth of degree log 2 (3). 



4.3. Substitutional Graphs. We give a self-contained description of the Schreier graphs S(G,P,S) for 
the examples above, in the form of substitutional rules. In this subsection, all graphs shall have a base point, 
and shall be edge-labelled; graph embeddings must preserve the labelings. 

Definition. A substitutional rule is a tuple (U, Pi, ... , R n ), where U is a finite d-regular edge-labelled graph, 
called the axiom, and each Ri is a rule of the form Xi — > Yj, where Xi and Y% are finite edge-labelled graphs. 
The graphs Xi are required to have no common label. Furthermore, there is an inclusion, written Li , of the 
vertices of Xi in the vertices ofYi; the degree of ii{x) is the same as the degree of x for all x € V(Xi), and 
all vertices of Yi not in the image of Li have degree d. 

Given a substitutional rule, one sets Go — U and constructs iteratively Gn+i from Q n by listing all 
embeddings of all Xi in Q n (noting that they are disjoint), and replacing them by the corresponding Yi. If 
the base point * of Q n is in a graph Xi, the base point of Gn+i will be £j(*). 

Note that this expansion operation preserves the degree, so Q n is a c?-regular finite graph for all n. We 
are interested in fixed points of this iterative process, or equivalently in a converging sequence of balls of 
increasing radius in the Q n , and call a limit graph (which exists by jjj) a substitutional graph. 

Theorem 2. For the five examples G described above, the Schreier graphs S(G,P,S) are substitutional 
graphs. 

As an illustration, here is the substitutional rule for the group T: 

t It 




5. Amenability and Spectra 



Let G be a group acting on a set X. This action is amenable in the sense of von Neumann J3j if there 
exists a finitely additive measure fi on X, invariant under the action of G, with n(X) = 1. 
A group G is amenable if its action on itself by left-multiplication is amenable. 

We now state the main connection between the spectra of our representations and dynamical systems. 
Recall 7r is the representation of G on L 2 (E N ) and 7r„ is the representation of G on L 2 (T, n ). Since n n contains 
7r„_i we write n n = 7r n _i ffi 7T^. 

Theorem 3. Let G be a group acting on a regular rooted tree, with ir, ir n and w„ be as above. 

1. it is a reducible representation of infinite dimension but splits as hq ® © n>1 tt^, so all of its irreducible 
components are finite- dimensional. Moreover spec(7r) = U n >o s P ec ( 7r ™)- 

However, if G is weak branch (see ^), then Pg/p is irreducible. 

2. The representations ir n and Pc/p n are equivalent, so their spectra coincide. The spectrum of Po/p is 
contained in U„>o spec(7r„), and therefore is contained in the spectrum ofir. 

If moreover either P or G/P are amenable, these spectra coincide, and if P is amenable, they are 
contained in the spectrum of pa ■ 

3. -Htt has a pure-point spectrum, and its spectral radius r{H 7T ) = s € R is an eigenvalue, while the spectral 
radius r(H PG/p ) is not an eigenvalue of H Pc/p . Therefore H Pa/p and are different operators having 
the same spectrum. 



6. Results on Spectra 



Since all the groups considered have intermediate growth, they are amenable and spec pc/p = spec7T. We 
now describe explicitly these spectra. For A S K let J(A) be the Julia set of the quadratic map z i— > z 1 — A: 



J(A) 



Group 


Spectrum of M 


Description 


© 


r 
r 


[-i,0]U[i,l] 
[0,1] 

{l,i}ui(l±J(6))} 

{i,-i J i}ui(i±y / |±2j(f)) 


two intervals 

nonnegative interval 

union of Cantor set of null Lebesgue 

measure and set of isolated points 

Cantor set of null Lebesgue measure 


F 




same as for T 



These computations imply the following results: 

Theorem 4. 1. There are connected 4-regular graphs of polynomial growth, which is are the Schreier 
graphs of groups of intermediate growth, and whose Markov operator's spectrum is any of the above 
sets. 

2. There are noncommutative dynamical systems generated by 3 (in the case of <&), 4 (in the case of<3) 
or 2 transformations, whose spectrum is any of the above sets. 

These spectra are all computed using the same technique: consider the representation ir n of dimension 
d n , given by d n x d n -permutation matrices 7r„(s), for all s £ S. These matrices satisfy block identities, for 
instance for the group 25 



7r n (o) 



l d r,- 

7r„_i(a) 

7T„_l(d) 



7Tn(&) 

ir n (d) 



7r„_i(a) 

7T n _l(c) 



7T„_l(6) 



Note that for our five examples 7r„(s) is expressed by blocks of the form 0, 1 and 7r„_i(s'). 
Define now the polynomial Q : C s+1 — > C by 



Q n ({X s }, A) = det ( x s*n(s) - A ) 



The spectrum of ir n is {A e C| Q n {-^, ■ ■ ■ , j^y, A) = 0}. Using the above block identities, it is possible to 
express Q n as rational expression over Q n -i, and therefore to compute inductively Q n and the spectrum of 
7r„. For the group 0, for instance, we have for n > 2, setting a = X a and /3 = X b = X c = X c i, 

ft. K ft A) . + 2A„ - (5^. A - + 3 ; A + -T-aO ■ 

In all our cases the polynomials Q n can be explicitly computed; again for the group 0, we have the 
Lemma 1. Define 

$o = a + 3/3 - A, <f>i = —a + 3/3 - A, 

$ 2 = -a 2 - 3/3 2 - 2/3A + A 2 , $„ = $ 2 l _ 1 - 2(2a) 2 "~ 2 /or n > 3. 

T/zen for all n G N we We Q„(a, /3, A) = $ $ x • • • $„ . 




On montre ci-dessus I 'ensemble des points d'annulation de Qe(X a = X a -i — o;, — X r -i — 1, A) pour 
le groupe T, et le graphe de Schreier S(T, Ps^a^ 1 ^r^ 1 ). 
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